Toward a Qualitative Theory of Safety Control - When and How to Panic Intelligently

Safe control of a physical system requires the ability to both detect and avoid dangerous situations, while striving to achieve performance goals. State-of-the-art controllers still tend to rely heavily on classical control theory with feedback, occasionally with some limited use of associational-reasoning (expert systems) to help detect threatening situations and recall standard recovery procedures [Dvorak, 1987] . The need for model-based reasoning to control in novel situations and with incomplete data is widely acknowledged, but largely unaddressed to date. Existing relevant work has tended to focus on model-based monitoring to track the system state over time, with little attention to reasoning about control itself. In such work, any provisions for finding safe control actions tend to be mixtures of associational-reasoning and general-purpose planning or qualitative simulation which suffer problems of brittleness and intractability, respectively. To address those shortcomings, I propose a qualitative theory of safety control. The goal is to make explicit the kinds of intuitions that human operators use to focus the task of avoiding dangerous situations . I view the problem of safety control as two major issues: when to worry and how to control. Intuitively, one often worries about the need to control only if the chance of danger has increased assuming the initial state was safe. Furthermore, different styles of control seem most appropriate in different types of situations. For example, when safety seems impossible to maintain it often seems best to try to at least delay the impending doom, in hopes that "buying time" might help . I argue that qualitative physics can be used to robustly capture such reasoning, but only with fundamental advances in how and what to qualitatively simulate .